Optimal regularization of linear and nonlinear perceptrons
نویسندگان
چکیده
We derive an analytical formula for the generalization error of linear adaptive classifiers trained with weight decay. Analytical and experimental results are then presented to analyze the optimal value of regularization parameters as a function of the training set size.
منابع مشابه
A numerical approach for solving a nonlinear inverse diusion problem by Tikhonov regularization
In this paper, we propose an algorithm for numerical solving an inverse non-linear diusion problem. In additional, the least-squares method is adopted tond the solution. To regularize the resultant ill-conditioned linear system ofequations, we apply the Tikhonov regularization method to obtain the stablenumerical approximation to the solution. Some numerical experiments con-rm the utility of th...
متن کاملEfficient estimation of multidimensional regression model using multilayer perceptrons
This work concerns the estimation of multidimensional nonlinear regression models using multilayer perceptrons (MLPs). The main problem with such models is that we need to know the covariance matrix of the noise to get an optimal estimator. However, we show in this paper that if we choose as the cost function the logarithm of the determinant of the empirical error covariance matrix, then we get...
متن کاملNonlinear Blind Source Separation by Variational Bayesian Learning
Blind separation of sources from their linear mixtures is a well understood problem. However, if the mixtures are nonlinear, this problem becomes generally very difficult. This is because both the nonlinear mapping and the underlying sources must be learned from the data in a blind manner, and the problem is highly ill-posed without a suitable regularization. In our approach, multilayer percept...
متن کاملSolving linear and nonlinear optimal control problem using modified adomian decomposition method
First Riccati equation with matrix variable coefficients, arising in optimal and robust control approach, is considered. An analytical approximation of the solution of nonlinear differential Riccati equation is investigated using the Adomian decomposition method. An application in optimal control is presented. The solution in different order of approximations and different methods of approximat...
متن کاملA regularization method for solving a nonlinear backward inverse heat conduction problem using discrete mollification method
The present essay scrutinizes the application of discrete mollification as a filtering procedure to solve a nonlinear backward inverse heat conduction problem in one dimensional space. These problems are seriously ill-posed. So, we combine discrete mollification and space marching method to address the ill-posedness of the proposed problem. Moreover, a proof of stability and<b...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 1994